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Abstract — This paper introduces tlie Reed Muller Sieve, a 
deterministic measurement matrix for compressed sensing. The 
columns of this matrix are obtained by exponentiating codewords 
in the quaternary second order Reed Muller code of length N. 
For k = 0{N), the Reed Muller Sieve improves upon prior 
methods for identifying the support of a fc-sparse vector by 
removing the requirement that the signal entries be independent. 
The Sieve also enables local detection; an algorithm is presented 
with complexity log N that detects the presence or absence 
of a signal at any given position in the data domain without 
explicitly reconstructing the entire signal. Reconstruction is 
shown to be resilient to noise in both the measurement and data 
domains; the £2/ £2 error bounds derived in this paper are tighter 
than the ^2/^1 bounds arising from random ensembles and the 
£i/£i bounds arising from expander-based ensembles. 

Index Terms — Deterministic Compressed Sensing, Model Iden- 
tification, Local Reconstruction, Second Order Reed Muller 
Codes. 



I. Introduction 

The central goal of compressed sensing is to capture at- 
tributes of a signal using very few measurements. In most work 
to date, this broader objective is exemplified by the important 
special case in which the measurement data constitute a vector 
/ = <I>a + e, where $ is an x C matrix called the sensing 
matrix, a is a vector in C'', which can be well-approximated 
by a fc-sparse vector, where a fc-sparse vector is a vector which 
has at most fc non-zero entries, and e is additive measurement 
noise. 

The role of random measurement in compressive sensing 
(see im and 121) can be viewed as analogous to the role 
of random coding in Shannon theory. Both provide worst- 
case performance guarantees in the context of an adversarial 
signal/error model. In the standard paradigm, the measurement 
matrix is required to act as a near isometry on all fc-sparse sig- 
nals (this is the Restricted Isometry Property or RIP introduced 
in 121). Basis Pursuit HI, H) or Matching Pursuit algorithms 
(H, 16] can then be used to recover any fc-sparse signal from 
the N measurements. These algorithms rely heavily on matrix- 
vector multiplication and their complexity is super-linear with 
respect to C, the dimension of the data domain. The worst case 
complexity of the convex programs Basis Pursuit HI, LASSO 
Q and the Dantzig Selector [8l is C'^ though the average case 
complexity is less forbidding. Although it is known that certain 
probabilistic processes generate N x C measurement matrices 
that satisfy the RIP with high probability, there is no practical 



algorithm for verifying whether a given measurement matrix 
has this property. Storing the entries of a random sensing 
matrix may also require significant resources. 

The Reed Muller Sieve is a deterministic sensing matrix. 
The columns are obtained by exponentiating codewords in the 
quaternary second order Reed Muller code; they are uniformly 
and very precisely distributed over the surface of an A^- 
dimensional sphere. Coherence between columns reduces to 
properties of these algebraic codes and we use these properties 
to show that recovery of fc-sparse signals is possible with high 
probability. 

When the sparsity level k = O ^\/iV^, recovery is possible 
using the algorithm presented in |]9l and the reconstruction 
complexity is only fc A^ log^ A^. The prospect of designing 
matrices for which very fast recovery algorithms are possible 
is one of the attractions to deterministic compressive sensing. 
When the sparsity level fc = 0{N) recovery is possible 
using the algorithm described in this paper Reconstruction 
complexity is NC, the same as for both CoSaMP l6| and 

ssMP m. 

We note that there are many important applications where 
the objective is to identify the signal model (the support of 
the signal a). These include network anomaly detection where 
the objective is to characterize anomalous flows and cognitive 
radio where the objective is to characterize spectral occupancy. 
The Reed Muller sieve improves on results obtained by Candes 
and Plan l?] in that for fc = 0{N) it is able to identify the 
signal model without requiring that the signal entries ai be 
independent. 

Reconstruction of a signal from sensor data is often not the 
ultimate goal and it is of considerable interest in imaging to be 
able to deduce attributes of the signal from the measurements 
without explicitly reconstructing the full signal. We show that 
the Reed Muller Sieve is able to detect the presence or absence 
of a signal at any given position in the data domain without 
needing to first reconstruct the entire signal. The complexity of 
such detection is A^^ log A^. This makes it possible to quickly 
calculate thumbnail images and to zoom in on areas of interest. 

There are two models for evaluating noise resilience in 
compressive sensing. We provide an average case error anal- 
ysis for both the stochastic model where noise in the data 
and measurement domains is usually taken to be iid white 
Gaussian, and the deterministic model where the goal is to 



approximate a compressible signal. It is the geometry of the 
sieve, more precisely the careful design of coherence between 
columns of the measurement matrix, which provides resilience 
to noise in both the measurement and the data domain. Our 
analysis points to the importance of both the average and the 
worst-case coherence. 

We show that the ^2 error in reconstruction is bounded above 
by the £2 error of the best fc-term approximation. This type 
of bound is tighter than the £2/^1 bounds arising from 

random ensembles IT], ||6l and the bounds arising from 

expander-based ensembles ifTDl . lfT2l . We emphasize that our 
error bound is for average-case analysis and note that results 
obtained by Cohen et. al. lfT3l show that worst-case ^2/^2 
approximation is not achievable unless iV = 0(C). 

II. Two Fundamental Measures of Coherence 

Throughout this paper we also abbreviate { 1 , • • • , C } by [C] . 
We shall use the notation Lpj for the column of the sensing 
matrix; its entries will be denoted by ipj (x), with the row label 
X varying from to — 1. We consider sensing matrices for 
which reconstruction of a is guaranteed in expectation only, 
and so we need to be precise about our signal model. 

A signal a € M'' is fc-sparse if it has at most k 
non-zero entries. The support of the vector a, denoted by 
Supp(a), contains the indices of the non-zero entries of a. 
Let TT = {tti,--- ,7rc} be a uniformly random permutation 
of [C]. Since our focus is on the average case analysis, we 
always assume that a is a fc-sparse signal with Supp(Q!) = 
{tti,-- • jTTfe} and the values of the fc non-zero entries of a 
are specified by k fixed numbers ai, • • • ,afe. We shall also 
define |aniin| = niin^:„,^o la^l- 

The following proposition is proved by Calderbank et. al 
191 , lfT4l and plays a key role in our analysis 

Proposition 1: Let a be a fc-sparse vector with support S = 
{tti, • • • , TTfe}. Let /i be a function from [C] x [C] to M, and let 
<!> be an X C sensing matrix. If the following two conditions 
hold: 

. (Stl). = max.^j \h{i,j)\ < N'"^ for < 77 < 0.5. 



(St2). v = max, 
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E,^.Mi,j) <^"^ for7>L 



Then for all positive e and for all fc less than min { | , } 



with probabiUty 1 
statements hold: 



k C cxp I — ^ I the following three 



(Spl) For every w in [C] — S": 

(Sp2) For every index i in {1, • • • , fc}: 



< ellall 



<e\\a\\ 



(Sp3) If C|a 



mmi^ > then 



Remark 1: A matrix satisfying conditions (Stl) and (St2) 
is called a StRIP-able matrix. Similarly a matrix satisfying 



conditions (Spl), (Sp2), and (Sp3) is called a StRIP matrix. 
Proposition [T] states that StRIP-ability is a sufficient condition 
for the StRIP property. 

III. The Reed-Muller Sieve 

Let m be an odd integer The measurement matrix <& = $m,r 
has 2'" rows indexed by binary m-tuples x and 2'^''+^)™ 
columns indexed by to x to binary symmetric matrices Q in the 
Delsarte-Goethals set DG{m, ?■). The entry fgix) is given by 
(Pq{x) ~ i^^^ , and all arithmetic in the expressions xQx^ 
takes place in the ring of integers modulo 4. The matrices in 
DG{m^r) form an (r + l)-dimensional binary vector space 
and the rank of any non-zero matrix is at least to — 2r (see 
ifTsl and also | fl4) for an alternative description). The Delsarte- 
Goethals sets are nested 



1 



DG{m, 0) C L»G(m, 1) d ■ ■ ■ d DG 



The set DG(m,0) is called the Kerdock set and it contains 
2™ nonsingular matrices with distinct main diagonals. The 
vector of length 2"' with entries xQx^ is a codeword in the 
quaternary Delsarte-Goethals code lfT6l . 

In Section [V] we will apply the following result on partial 
column sums to guarantee fidelity of reconstruction. 

Proposition 2: Let V and W be two binary symmetric 
matrices and let Mw and Mv-w be the null spaces of 

and y - ly. If S = Y^^^^^aVa^ +xWx'' +2aWx'^ ^ 

|g2| _ 2l-'^«'l+l-'^''-"'l 
Proof: We have 

aVa'^ +bVb'^ +xWx'^ +yWy'^ +2aWx'^ +2bWy'^ 



a^h,x,y 

Changing variables to z 



y, c = a + b, y and b yields 



cVc ' +zWz ' +2cWz 



j^^(_l)(cv+..w).-^ (E(-1) 



{zW+dw+cW}y^ 



The terms in Equation ([T]) vanishes unless cV + dv + zW = 
and zW + dw + cW = simultaneously. Hence, we can 
rewrite Equation ([T]) as 



E 



{c+z)W{c+z) ' +ciV~W)c 



c.z 

{c+z)W=dw 
c(V+W)=dv+div 

Write c = ci + e with ciiV + W) = dy + dw and e{V + 
W) = 0, and c + z ^ (c2 + Z2) + f with (c2 + Z2)W = dw 
and fW = 0. Then 



E 



fWf 



E 



e(V-W)e^ 



22m 2\^w\ + \J^v-w\ 



Proposition [2] bounds the worst case coherence between 
columns of ^ni,r- We bound average coherence by dividing 



the columns into a set H indexed by the matrices in DG{m, r) 
with zero diagonal, and a set D indexed by the matrices in the 
Kerdock set. The columns in H form a group under pointwise 
multiplication. 

Lemma 1: Let $ be a DG{m, r) sensing matrix. Then 



-N 



1 



for every index i. 



Proof: Any column can be written as a pointwise product 
hd with /i in H and d in D. Average coherence with respect 
to hd is then 



'h'd'. 



(2) 



If d ^ d', then h' ranges over all elements of H and 
Y,h' h'^^' = 0- Otherwise h' h and Y^w^h ^^^h' = -1. 
In this case d~^ld ~ N, which completes the proof. ■ 
The normalized Delsarte-Goethals sensing matrix is given by 
$ = ^^'^ have now proved 

Theorem 1: The normalized matrix $ satisfies Condition 

2 ml ■ 



(Stl) with rj = 1+r and Condition (St2) with 7 



and 



A. Noise Shaping 

The tight-frame property of the sensing matrices makes it 
possible to achieve resilience to noise in both the data and 
measurement domains. Note that the factor that appears in 
Lemma |2] can be reduced by subsampling the columns of 

Lemma 2: Let <r be a vector with C iid N{Q,cr^) entries 
and e be a vector with TV iid A/'(0,(T^) entries. Let fi, = 
and u = h + e. Then u contains entries, sampled iid from 
A/" (0, (T^), where cr^ = -^cr^ + (t^ and with probabiHty 1 — ^, 
||m|| < ^/NlogCa. 

Proof: Each element of h is an independent Gaussian 
random variable with zero mean and variance at most 
Hence, each element of u is a Gaussian random variable with 

zero mean and variance at most ■w'^Ji+'^m- It therefore follows where c is a constant. 



Chirp reconstruction identifies the signal model (the support 
of the k significant entries) by analyzing the power spectrum 
of the pointwise product of the superposition / with a shifted 
version of itself. The Walsh-Hadamard transform of this point- 
wise product is the superposition of k Walsh functions and 
a background signal produced by cross-correlations between 
the k significant entries and cross-correlations between these 
k entries and noise in the data domain. We shall prove that 
the energy in this background signal is uniformly distributed 
across the Walsh-Hadamard bins, and that with overwhelming 
probability this background bin energy is sufficiently small to 
enable threshold detection of the k tones. We show that sparse 
reconstruction is possible for k = 0{N) by averaging over all 
possible shifts. Note that the original chirp reconstruction al- 
gorithm analyzed in ifTTI has minimal complexity k N log^ A^ 
but reconstruction is only guaranteed for k = O (^^/N^. Our 
main result is the following theorem. 

Theorem 2: Let $ be an A^ x C normalized DG{m, r) 
matrix. Let a be a fc-sparse vector with uniformly random 
support contaminated by Gaussian white noise with variance 
(T^ . Let / = $a + e, where the measurement errors are white 
Gaussian with variance ct^. Then if fc < |- and 



361ogC||aj| 



(3) 



(4) 



then with probability 1 — §, chirp reconstruction recovers the 
support of a, and furthermore. 



|$(a- 



< cfclogC 



N 



(5) 



from the tail bound on the maximum of A^ arbitrary complex 
Gaussian random variables with bounded variances that 

-1 

Pr 



> v/2cr2 logC < 2 (^27rlogCC <C 



The fast Hadamard transform is used to calculate the power 
spectrum across all A^ Hadamard bins. Each bin i has the 
value 



1 ^ 

^ x—1 



f{x + a)f{x) 



(6) 



IV. The Chirp Reconstruction Algorithm 



Algorithm 1 Chirp Reconstruction 



for i = 1, • • • A^ do 

Choose the next binary offset a. 
Pointwise multiply / with a shifted version of itself. 
Compute the fast Hadamard transform: [f). 



For each A e [C], calculate Aa.o = 



end for 

For each A e [C], take the average of Aa over all Aa.o- 
Let 5* be the position of the k highest (in magnitude) 
average peaks. 
Output a = ($t;$s)"^$t;/. 



Given the offset a, evidence for the presence or absence 
of a signal at position delta in the data domain resides in 
the Hadamard bin £ = qQa- After aligning the phase, the 
final step is to average over all offsets a. The notation Ea 
emphasizes that the average is taken over all offsets. The 
following theorem shows that Ea [Aa(j/)] consists of k distinct 
Walsh tones staying on top of a uniform chirp-like residual 
term. 

Theorem 3: Let u = + e denote the overall noise. Then 
as long as fc < ^, with probability 1 — ^, for every index A 
in [C] 

Ea [AA(/)]=^^<5A,.,+i?^(/), 

2—1 ^ 



where R'^{f) consists of the chirp-Hke and signal/noise cross 
correlation terms, and 

9VbiC||a||2 9y^hiC\\a\\\\u\\ 



Ni- 



(7) 



Theorem|3]is proved in SectionlV] The next lemma provides 
a lower bound on the number of required measurements. 

Lemma 3: Let <r and e be the white Gaussian data and mea- 
surement noise vectors with variances cr^ and cr^^ respectively. 
Let = al 



C_ 2 Tf I, < c 



^ 36VlogC||Q|| 



and a < 



361ogC||a|| 



then with probability 1 — ^, chirp reconstruction successfully 
recovers the positions of the k significant entries of a. 

Proof: Chirp detection generates k Walsh tones with 



magnitudes at least 



above a uniform background signal. 



Furthermore, with probability at least 1 — every background 
signal at every index is bounded by the right hand side of (|7]i- 
Hence, if the right hand of ^ is smaller than then 
the k tones pop up and we can detect them by thresholding. 
Hence, we need to ensure that 



9||a!|2VT^ ^ |a„,i„| 



< 



4ViV 



and 



9715^11 



< 



Now Lemma |2] states that with probability 1 — i, 
llujl < \/N\ogC (J. Consequently, to provide successful sup- 
port recovery we need to assure that 



9ViV logC||a||CT 



< 



4ViV 



Proof of Theorem ^ Lemma [5] guarantees that with 
probability 1 — § Chirp Detection successfully recovers the 
support S of a. We then approximate the values of a by 
regressing / onto S. By Lemma |2] without loss of generality 
we can assume that a is exactly fc-sparse and the measure- 
ment errors are white Gaussian with variance a^. We have 
||$q: — $ci||^ < llT'swII^, where Vsu denotes the projection 
of the noise vector onto the space spanned by $s. Now it 
follows from the Gaussian tail bound (See ifTSl ), that with 



This £2/^2 error bound for average-case analysis stands in 
contrast to results obtained by Cohen et al ITSll showing 
that worst-case £2/^2 approximation is not achievable unless 

N = 0{C). 

Remark 3: There are many important applications where 
the objective is to identify the signal model (the support of 
the signal a). Note that in contrast to Q chirp reconstruction 
does not require that the component signals ai be independent. 
When the N x k submatrix is well conditioned the approxi- 
mation bound (|5]l in the measurement domain (||<i>(Q; — a)\\) 
can be translated directly to approximation bound in the data 
domain (jla — a\\). 

Remark 4: Chirp reconstruction is able to detect the pres- 
ence or absence of a signal at any given index A in the 
data domain without needing to first reconstruct the entire 
signal. The complexity of detection is log N. If the signal 
a were the wavelet decomposition of an image, then chirp 
reconstruction can be applied to the measured signal to recover 
thumbnails and to zoom in on areas of interest. 

V. The Sieving of Evidence 
We now prove Theorem|3]in the special case where there is 

no noise (it = 0). We begin by expanding / as X]i=i Q^i'Piri 

where (fTnix) = I'^Q^i'^ . 

Lemma 4: Let $ be a normalized N x C sensing matrix. 

Then for any offset a and Hadamard bin £ in F™ 

|2 



r^a(/) = E 



N2 



(8) 



where Raif) is the quantity 



1=1 

Next we analyze the power of chirp reconstruction to detect 
the presence of a signal at some index A. 

Lemma 5: If A is any index in C then after Step |5] of Chirp 
Reconstruction, for all A in [C] 

Ea [AAmi-E^-^A.^.+i^^a), 



probability I - h, ||Psu|P < cfclogCa^ (where c is a where 



constant). Therefore 



C 



!<!>«- <l>a||^<cfclogC (a4 + -a^ 



Remark 2: We have focused on stochastic noise but we 
have derived similar results for the best fc-term approximation 
ai_i.fc in the context of the deterministic noise model (see 
lfT4l ). By combining the Markov inequality with Theorem [T] 
we have shown that for every 5' 



N 

'i— 1 

i?^(/)|<9-^ 



, and 

l|2 



Proof: If A = TTi for some i then the signal ai contributes 



■L-^ to the Hadamard bin a Qa- Rotation by i 



and averaging over all offsets a accumulates evidence ^ for 
the presence of a signal at the index A. 

If A 7^ TTi for any i, then it is only the cross-terms 
that contribute to the Hadamard bin qQa- Rotation of the 
contribution Ra^^if) by i-°-Q^°- and averaging over all 
offsets a produces the background signal against which we 
perform threshold detection. 



Pr 



\u\\2 > ||e||2 



1 



<6'. 



Define h{-Ki^Tij) by 



VII. Conclusion 



ja(Q^,-QA)a^ '^^^_l^{aQ^,-aQ i^)x'^ jx{Q^ .-Q^ .)x'^ 



We will bound by applying Proposition [T| so we need 

to verify Conditions (Stl) and (St2). By Proposition |2] the 
expectation above is always bounded in magnitude by the term 
2^''. Hence it remains to bound average coherence, and here 
we show that max^ \&j^ih{i,j)\ < jr^. We rewrite h{TTi,TTj) 
as 



Note that as a ranges over the finite field F™, a + x also 
ranges over F™. Therefore i(''+^)(Q->-QA)(a+x)T ^ 
constant column sum, independent of the choice of j, and 
has magnitude smaller than N. As a result 



Lemma [U then implies that 



J2^x{Q^-Q^^)x^ 



< 



C-1' 



We have now shown with respect to h that $ satisfies 
Condition (Stl) with 77 = 1 + r and Condition (St2) with 
7 = — — . It then follows from applying Proposition [T] with 



e = N^9^/\ogC, 
that with probability 1 — g-. 



1 



is bounded by 9A^™"2 jjalpVlogC- ■ 

VI. Noise Resilience 

When noise is present in the data domain or in the measure- 
ments, the power spectrum contains extra terms arising from 
the signal/noise cross correlation and noise autocorrelation. It 
is natural to neglect noise autocorrelation and to focus on the 
cross correlation between signal and noise. 

Let y = $a. At the end of Step |7] for each index A, the 
signal/noise cross correlation can be represented as 



E„ 



N 



y{x)u{x + a){~l)'"^ 



(9) 



We have modified the argument used to prove Lemma |5] 
to show that with probability 1 — 77, the signal/noise cross 
correlation term is uniformly bounded by 9Af~~2||a||||u|| 
(see 1141 for more details). 



In compressed sensing the entries of the measurement vector 
constitute evidence for the presence or absence of a signal at 
any given location in the data domain. We have shown that the 
Reed Muller sieve is able to identify the support set without 
requiring that the signal entries be independent. We have also 
demonstrated feasibility of local decoding where attributes 
of the signal are deduced from the measurements without 
explicitly reconstructing the full signal. Our reconstruction 
algorithms are resilient to noise and the £2/(^2 error bounds are 
tighter than the £2/^1 bounds arising from random ensembles 
and the £i/£i bounds arising from expander-based ensembles. 
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